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ABSTRACT 

This paper presents the design and development of 
Tactile Interfaces with Enhanced Images with Patterns 
and Textures for Visually Impaired People. In 
particular, haptic interfaces uses a number of 
predefine tactile texture and patterns to enhance the 
experience of the tactile feeling of images for visually 
impaired people. The open source OpenCv application 
is used to automatically change group of colours from 
depth images to tactile texture and patterns. The depth 
images are captured with the heXp of RGB-Depth 
cameras. The tactile images are then explored by 
visually impaired people with the use of haptic 
interfaces. The predefined textures and patterns are 
more easily distinguished than uniformly colors in 
depth images. 

Key Words: Haptics, tactile interfaces, 3D models, 
depth images, RGB-Depth cameras, haptic geometric 
patterns 

1. INTRODUCTION 

Touch is a basic sensation for the understanding of 
our surrounding environment. The sensation of touch 
involves the skin's ability to separate the surface 
properties of an object, and the kinaesthetic ability of 
the muscles to determine the body's position. 

The reaction that we acquire whenever we explore the 
shape, the texture, the geometry, the mass, the 
elasticity, and the dimensions of objects with our 
hands is called tactile feedback. There is also the 
kinaesthetic feedback which is the stimuli that is 
produced in muscles and tendons from power-force 
translation, limb motion control, and body position. 

Visually impaired people are depending in the tactile 
and kinaesthetic feedback to explore their 


surroundings. RGB-Depth cameras can record the 3d 
model of the environment and replicate it to a 3d 
virtual reality. Haptic interfaces can be used by 
visually impaired people to explore and react with this 
augmented virtuality. Geometric patterns can be used 
to increase the experience of the haptic interface user 
and enhance the haptic feeling of the 3d objects. 

The rest of the paper is organized as follows: Section 

2 presents a related work on tactile interfaces. Section 

3 presents the 3D depth monitoring technology that is 
used to capture and reproduce the 3D virtual model of 
the surroundings. Section 4 proposes the geometric 
patterns that could be used for the tactile enhancement 
of the 3D models. Section 5 describes the open 
OpenCV algorithm that manipulates the 3d models to 
reproduce the enhanced tactile 3d objects. Section 6 
describes the H3D haptic program that transforms the 
patterned image to haptic objects. Section 7, finally 
concludes the paper. 

2. HAPTIC DEVICES 

Virtual objects acquire realistic properties such as 
mass, hardness, texture and elasticity. The tactile 
interface is an interactive actuator that interacts with 
the 3D object and the user in a virtual environment. It 
receives commands from the user and manipulates the 
3D virtual objects. It consequently receives feedback 
from the virtual environment and creates stimuli on 
the user’s skin and muscles. 

The tactile interfaces are split into two categories 
according to the stimuli that they produce to the users. 
They are divided into tactile interfaces when the 
stimuli that they produce refers to the skin of the user 
and to kinesthetic haptic interface when the stimuli 
refers to muscle of the user and his/her body position. 
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Tactile Interface are more common and they support 
skin feedback and produce vibration, pressure and 
heat to the users fingertips. They help the user to 
explore elasticity, the texture and the dimensions of 
virtual 3D objects. 

Kinesthetic haptic devices provide feedback to user 
according to his movements and his body position. 
They help the user to explore the mass, the friction 
and the hardness of the virtual 3D objects. 

One commonly used haptic devices, is the “3D 
Systems Touch Haptic Device” (1), figure 1. It offers 
both tactile and kinesthetic feedback to the user. It is a 
haptic device that user a ring or a stylus to interact 
with the user. It measures the position of the user's 
fingertip. The user can feel the sense of touch, 
movement constraints, friction and surface texture, 
and other features of virtual objects. 



Figurel. 3D Systems Touch Haptic Device 

Another common haptic device is the“Novint Falcon” 
(2). It was released in 2007 and it was the first haptic 
device that offered high resolution 3D feedback to the 
user, Figure 2. 



Figure2. Novint Falcon Haptic Device 

The 3D Systems Touch Haptic Device offers six 
Degrees Of Freedom (DOF) to the user while the 
Novint Falcon offers 3 DOF. Both devices uses 1kHz 
servo rate, which means that they can produce 1000 
update packets per second. 


3. DEPTH CAMERAS FOR 3D MODELLING 

In order to replicate real world objects to 3D virtual 
objects, RGB-depth cameras are used. One low cost 
RGB-depth camera that is commonly used is the 
Microsoft Kinect Depth sensor (3). It is composed by 
a VGA video camera, a CMOS depth sensor and an 
infrared projector. The projector emits infrared light 
to the physical object. The physical object reflects the 
infrared light back to the CMOS depth sensor and the 
depth sensor captures the reflected infrared light. The 
time passed from the moment the projector emits the 
light until the light reaches the depth sensor defines 
the distance between the sensor and the physical 
object. This time is called “time of flight”. 

In order to connect the kinect sensor to the pc and 
produce custom software for 3D-Depth mapping, 
Microsoft provides the Kinect Software Development 
Kit (SDK) and the Kinect Studio. The kinect studio 
captures the 3D image, the VGA image and the depth 
images, as shown in Figure 3. 



Figure3. Images captured from the Kinect sensor. The 
VGA image at the left side, the Depth image at the 
middle and the 3D image at the right 
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4. MAPPING DEPTH IMAGES TO ENHANCED PATTERNED IMAGES. 

In order these images to be more easily haptically explored the depth images of figure 3 should be transformed 
to a grey scale image as shown in Figure4. 



Figure4.Depth images transformed to gray scale images 


Let us assume that we have an 11 RGB Grey scale, as depicted in figure 5. 


No. 

Red 

Green 

Blue 

RGB % 

Hex Color Code 

1 

255 

255 

255 

(100,100,100) 

FFFFFF 

2 

229 

229 

229 

(90,90,90) 

E5E5E5 

3 

204 

204 

204 

(80,80,80) 

CCCCCC 

4 

178 

178 

178 

(70,70,70) 

B2B2B2 

5 

153 

153 

153 

(60,60,60) 

999999 

6 

127 

127 

127 

(50,50,50) 

7F7F7F 

7 

102 

102 

102 

(40,40,40) 

666666 

8 

76 

76 

76 

(30,30,30) 

4C4C4C 

9 

51 

51 

51 

(20,20,20) 

333333 

10 

25 

25 

25 

(10,10,10) 

191919 

11 

0 

0 

0 

(0,0,0) 

000000 


Figure5. Gray scale in the RGB model 



These 11 Grey levels should be corresponded to the simple geometrical patterns, such as those of Figure 6. 




\\\\\\\\ 
//////// 
\\\xWW 



+ 

+ 

4 

4 

4 

4 

+ 

+ 

4 

4 

4 - 

4 

+ 

4 

+ 

4 

+ 

+ 

+ 

+ 

4 

4 

+ 

+ 

+ 

4 

4 

4 

4 

4 

4 - 

4 - 

+ + 

+ 

4 - 


Figure6. Simple Geometrical Patterns for enhancing tactile interfaces 
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The image of Figure 4 is then transformed in Figure 7. 



Figure7. Haptically enhanced depth image 


5. THE USE OF OPENCV FOR AUTOMATIC 
TRANSFORMATION OF RGB IMAGES TO 
HAPTIC IMAGES 

This transformation can automatically be done with 
the help of the OpenCV (4) image processing library 
and python. 

The imread function will load the colored RGB image 
to OpenCV. Next, we convert the color image to 
grayscale with the help of the cvtColor function. 

In order to convert the original image from the RGB 
color space to gray, the code COLOR_BGR2GRAY 
is used. 


stiffness and the maximum height of the objects can 
be given to objects with the following XmL code. 

<DepthMapSurface stijfness="0.6" 
maxDepth = "0.008" 
staticFriction ="0.9” 
dynamicFriction= "0.9 " 
whiteIsOut="False" > 

<ImageTexture containerField= "depthMap" 
url= "patterned_image.png" repeats = "false" 
repeatT = "false "/> 

</DepthMapSurface> 


Gray = cv2.cvtColor 
(Image, cv2.COLOR_BGR2GRAY) 

Now we have to correspond the levels of the gray 
scale of Figure 5 to the geometrical patterns of Figure 

6. This is done with the help of the library colorspace. 

6. TRANSFORM GRAYSCALE IMAGES TO 
HAPTIC OBJECTS 

In order to haptically explore the patterned images 
produced in the previous section the H3D haptic 
application is used (5). The H3D uses XML files to 
pass haptic properties to the objects illustrated in the 
pictures. The static friction, the dynamic friction, the 


7. CONCLUSION 

The system proposed gives the ability to visually 
impaired people to haptically explore images and the 
surrounding environment with the help of tactile 
interfaces. Predefined patterns and textures are used to 
enhance the experience of the user. Predefined 
textures and patterns help visually impaired people to 
distinguish more easily the objects that are included in 
a picture. The color of the object is changed 
automatically with the help of the OpenCv application 
to the predefined patterns. These patterns are more 
easily distinguished by haptic interfaces than 
uniformly colors. 
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